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ftBSTBACT 

The first year of the writing program at the William 
M. Trotter Elementary School (Boston, Massachusetts) accomplished (1) 
the development of an outline of basic writing activities and skills 
for grades one through five: (2) the use of holistic rating; (3) the 
development of a common correction symbol: (U) the development of 
writing folders containing student writing samples; (5) the 
establishment of a central file of ideas and materials; (6) in-class 
demonstrations of activities to develop writing skills; and (7) 
meetings with cr communications to parents about the writing program. 
Teachers' evaluations indicated that the children enjoyed writing 
more^ that the frequency of writing practice increased, and that the 
students* writing appeared more organized. On the basis of one 
holistic rating session of one set of grade five writing samples, it 
was determined that the writing program had increased the writing 
ability of the poorest writers in that grade. (Attachments include 
directions for eliciting writing samples, criteria for judging 
writing samples, outline of the curriculum guide, the correction 
symbol system, and samples of student writing.) (HOD) 
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EVALUATION OF THE WRITING PROGRAM 
AT THE 

WILLIAM M. TROTTER SCHOOL 
BOSTON, MASSACHUSETTS 

September, 1978-January , 1980 



Sandra Stotsky, Ed.D. 
Curry College 

Introduction 

During the academic year of 1977-1978, parents, teachers, and administra- 
tors of the William M. Trotter School Initiated discussions with members of the 
Education Faculty at Curry College to establish a pairing. The representatives 
of the Trotter School suggested that the development of a writing program for 
the entire school was their first priority, Dr, Sandra Stotsky, a member of 
the faculty of Curry College, was asked to serve as the College Coordinator for 
the pairing and to help the school develop a more organized and effective 
writing program. 

First Year Objectives 

The following objectives were stated in the Chapter 636 proposal for 1978-1979i 

1, Development of an outline of basic writing activities and skills for each 
grade level, 1-5- 

2, Understanding and use of holistic rating of writing samples, 

3, Development of a common correction symbol system, 

4, Setting up of writing folders for each student containing samples of differ- 
ent kinds of writing, 

5* Establishment of a central file of ideas and materials. 

6, In-class demonstrations of activities to develop writing skills by the 
College Coordinator and her student teachers, 

7. Meetings with or communications to parents about the writing program. 

Evaluation of First Year Objectives by Teachers 

On March 26, 1979f all teachers evalmted the restalts of the first year 
of the program by means of a questionnaire. Table 1 summarizes results from 
one part of this questionnaire. Most teachers felt that first year efforts to 
achieve most of 'Uiese objectives were satisfactory. 

When asked to comment upon the effect of the first year of the writing 
program upon the children's motivation to write, ten teachers stated that, in 
their judgment, the children enjoyed writing more, were more motivated and con- 
cerned for quality, and displayed more Interest in writing. Ten teachers also 
Indicated that the frequency of writing practice increased in their classrooms 
as a result of the program, in some cases to a dally basis. When asked to 
judge the effect of all the activities flowing from the fiiost year of the pro- 
gram upon students* writing skills, twelve teachers responded that their 
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Table 1 



Evaluation of Fi^st Year of Writing Program. by Trotter School Teachers * 



First Y^sar Objectives 


Satisfactory 


Unsatisfactory 


Unable to 
Judge 


i. Development of a chaxrt of basic 
• written language activities or 
skills for each srade IpvpI 


Lb 




2 


\ 

2. Understanding and use .of holistic 

ratins of writing Qamnl AO 


14 


Z 


2 


. 3. Development of a common correction 
symbol system. 






2 


4. Setting up of x^riting folders for 
each student containing samples of 
u^^^cLCLiu i^iuQs o£ wrxbXng* 


• 

17 


- 

1 




5. Establishment of a central file of 
ideas and materials. 


6 


2 


11 


6. In-class demonstrations of 'activi- 
ties to develop writing skills by 
the Curry College Coordinator and 
student* teachers 


11 




K 


7. Meetings vrith or communications to 
parents about the writing program. 


3 




13 


Proposed Second Year Obiectives 

1. Use of the chart to see if it is workable, comprehensive. 
1 and flexible. 

f : ■ ^ 


Agree 
17 


Disagree 
1 


. 2. Identification and collection of available materials for 
teaching or reviewing skills listed on chart. 


16 


2 


**3. Development of a sequence for teaching handwriting skills 
with kindergarten teachers. 


15 


1 


4. Further experience in doing holistic rating of writing. 
• a : 1 


12 


6 

- 



lab^e*! (continued) 



Agree Disagree 



5. Working oufc ways to teach students how to use this technique. 


12 


5 


6. Review of correction symbol list to make necessary revisions 
deletions, or additions for more able writers. * 


15 


3 


7. Improvement of the organization and use of writing folders. 


16 


2 


8. Further building up of central files of ideas and materials 
. . by pods, with a list of materials available for each taacher. 


12 


2 


.9. Use of a multi-ethnic and multi-racial theme to be encouraged 
on a monthly basis to promote both written and oral expressior 


11 


6 


10. Collection of activities for developing listening and speaking 
skills for central files. o f e 


18 


- 


11. Demonstration of methods and materl ^s by College Coordinator 
and student teachers to develop writing skills in the content 
areas, specifically learning how to take notes, use reference 
materials, outline, and develop paragraphs. 


15 


3 


12. Demonstration of methods to develop speaking and listening 
skills through writing activities. 


16 


2 


13. Continuation of Currv nolleoe'c oni i 

'..•.^wt. wj. viu<.<.jr s coxxaDoration with the 

Trotter School and its efforts to keep parents informed of 
the progress of the writing program. 


13 


5 



«ader?l ; ?r J"^'^"" ^""^^ School, 7 teachers in the primary 

grades (1 to 3) and 11 teachers in the elementary crade«. ?q ^r, «;> tv,» C P'^^'^^^^ 

■ each cell indicates the total number of teachers gifi^f^hat rating. 

** Sree!'"'"'' evaluation of this item, two of the three kindergarten teachers checked 
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students* writing definitely appeared better, more organiized, less careless, 
ma±e flaent, and more grammatically and mechanically correct, A number of 
teachers felt that the use of correction symbols and dictation exercises had 
fostered much more self-correction and saw improved self-correction as one 
of the most important achievements of the program. 

Objectives agreed upon for the second year of the program, 1979-1980, are 
also indicated in Table 1. 

Holistic Evaliiation of Writing 

During the Pall of 1980, Dr. Ronald Nut tall. Chapter 636 evaluator, and 
Dr. Stotsky designed a way of evaluating the effects of the writing pwgram by 
using a holistic scoring method. A control group was obtained by pulling from 
the files a set of compositions written by fifth grade students in the Spring 
of 1978, before the program began. The experimental group was a set of com- 
positions on the same topic, produced by students currently attending the fifth 
grade. The teachers gave identical directions to each group (see Attachment A). 

1. The assumption was made that the student population of the Trotter 
School had not changed In the two years and the Grade 5 children in 1978 were 
similar to Grade 5 children in 1980. Writing samples collected by Dr. Stotsky 
from Grade 5 children in June, 1978, were considered the control group papers. 
The writing program had not yet begun and these papers reflected the writing 
ability of Grade 5 students at the Trotter School at the end of whatever 
constituted their writing experiences at the school. Writing samples written 
by current Grade 5 students in response to the same topic were collected in 
JaJiuary, 1980. These papers were considered to be the exrv^rlmental group papers. 

2. One holistic rating session was planned; both control and experimental 
grfSHp^ papers would be jrandomly mixed together with no identifying data. Under 
the null hypothesis of no difference between the experimental and control 
conditions, it should be expected that the holistic scores for the compositions 
should be distributed similarly. 

3. It was hypothesized that the experimental group of Grade 5 students, 
who had for the most lart been at the Trotter School during Grade 4 (the first 
year of the writing program) would, after only ^ months woik in Grade 5i write 
papers as good as those written by the control group students after 10 months 
of Grade 5 woik in 1977*1978. 

The following procedures were used for the evaluation: 

1. Forty-ei^t Grade 5 papers were available from June, 1978. All identi- 
fying data were removed, and all papers were numbered and coded. 

2. Forty-ei^t Grade 5 papers were randomly drawn from the I98O samples. 
All identifying data were removed, and all papers were numbered and coded, 
and then piaced together randomly with the control group papers. 

3. Dr. Stotsky read throu^i all 96 papers and selected 12 papers for -^:he 
training session. These papers were selected to represent the total range of 
writing on this topic. She independently rated these papers on a 1 to 4 scale. 
(Ideally, she should have had an assistant do a second independent rating to 
check against her own ratings. Unfortunately, this was not possible at this time.) 
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4. Riotocopied copies of these 12 papers were prepared for the 2^ members 
of the Trotter School staff who were to psurticipate in the training session. 

5. At the training session on January 14, 1980 , the Trotter School staff 
spent about one hour rating and discussing their criteria for rating these 12 
papers. 

6. Pr. Stotaky later prepared a guide sheet for rating these Grade 5 
samples, based on the criteria stated by the teachers during their training 
session. These criteria are presented in Attachment B. She also prepared 
rating sheets for the teachers to use and grouped all the papers in batches of 10, 

7. On Januaiy 25, 1980, 9 upper elementary grade teachers participated in 
a 30-mlnute rating session. All teachers rated approximately 30 .^pers, and 
each paper was rated by 3 f.^rfe^rent teachers so that any discrepancies (more 
than a 1-polnt difference) T;; tween the first two raters could be resolved by 
the third rater's score, Orjiy 5 discrepancies appeared in $6 pairs of ratings. 



8. Papers were late: 
experimental group. 



.ecoded to distinguish the control group from the 



Results of the Holistic Evaluation of Writing 

Table 2 shows ihe scores of the two groups of papers. The papers were 



rated on a 1 to 4 scale by two separate raters, 
added together to give a total score range from 



These two raters' scores were 
2 to 8. The best papers wil 



have been scored 7 or 8, the worst, 2 or 3. 
Table 2. 



These results are presented in 



Table 2 

Distribution of Ratings from the Holistic Evaluation of Grade 5 
Writing Samples from Students at the Trotter Sphool 
in June, 1978 and January, 1980 



SAMPLE 



SCORE 



EXIERIMENTAL 
(January, 1980) 

CONTROL 
{June, 1978) 



TWO 



THREE 



12 




9 



FIVE SIX SEVEN EIGHT 
11 9 5 2 

5 U 3 2 



To examine these data statistically, the Kolmogorov-Smimov Two-Sample 
Test was used. This test indicates whether two independent samples have been 
drawn from the same population, or from populations with the same distribution. 
The test is concerned with the comparison of two cumulative distributions. 
Since there were 48 papers in each sample, we can consider each paper to 
constitute about 25? (2.083& to be more exact) of the total. Table 3 presents 
the data in Table 2 in the form of cumulative distributions. 
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Table 3 

Data from Table 2 Presented as Cumulative Distributions 



SAmE SCORE 





WO 


THREE 


FOUR 


FIVE 


SIX 


SEVEN 


EIGHT 


SXIERIMEirPAL 




\% 






8^ 




100?$ 


G0NTRC3L 




JTfo 




679S' 


905^ 




100?g 


DIPFEHENCE(E-C) 




-2^ 


-12^ 




-55 


Ofo 





It can be seen that the largest difference in the cumulative distribution 
occurs at a rating of ••3", with tha difference some 2755. The Kolmggorov- 
Smimov Two-Sample Test uses this largest difference between the two cumula- 
tive distributions to. produce a Chi -square statistic. The formula for the 
One-tailed test ist 

CHI-SQUARE '-^ {k) (D) (D) (nl* n2)/(nl n2) 

In this formula D is the largest difference between the cumulative distribu- 
tions, expressed as a proportion j nl is the number in the first group, n2 is 
the number in the second group • 

In this case the value is: 

CHI-SQUABE = ^♦(.27) (.27) (48»if8)/(48fW) 

CHI -SQUARE 37 with 2 degrees of freedom. 



This is hi^ly significant, beyond the .05 level of pro^bability. The 
conclusion is that the EXEERIMENTAL group of papers, written in Janxiary of I98O 
by Grade 5 students, were better than the COHTROL-* group of papers, written in 
June, 1978 by Grade 5 students. A careful examination of the data indicates 
where the difference is. In 1978, some 18 or 37.56 of the papers were scored 
as V2" or "3", the two lowest possible scores. In January of 1980, only 5 or 
10% of the papers were scored this low. Thus, results of this holistic rating 
session indicate that the bottom of the distribution has been raised. 

Conclusion 

On the basis of this one holistic rating session of one set of Grade 5 
writing samplest it is possible to suggest that the writing program has increased 
the writing ability of the poorest writers in Grade 5* However, this conclusion 
must be tentative; we would need to rate in the same manner at least several 
sample papers from each child before we could be more certain that the writing 
of the poorest writers in 1979-1980 is indeed better than the writing of the 
poorest writers in 1977-1978. 
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It is also possible to s^iggest that the benefits of the writing program 
extendad to the other students as well. The other students wrote papers at 
the end of only 5 months of the current Grade 5 program as good as those 
written by Grade 5 students in 1978 at the end of the school year. This indi- 
cates that the gains of the poorest writers in 1979-1980 have not been 
achieved at the expense of the other students. In otiier words, while the 
writing program has significantly helped the poorest writers, it has helped 
the others to write better too. 

If the results from this one rating session are generalizable, what 
could account for the improvement in student writing in 1979-I98O? Clearly, 
the Trotter School faculty deserves the credit. Teachers in their assessment 
of the writing' program at the end of its first year indicated that much 
more writing was taking place; they were more aware of the need for more frequent 
writing and were more willing to assign more writing. Teachers were also using 
techniques (such as the correction symbols) to encourage greater student res- 
ponsibility for correcting surface errors. Students may now feel that writing 
is a more meaningful and valued part of the curriculum. Certainly, an increase 
in practice alone may well be the key factor in the initial improvement of 
writing skills in the poorest readers and writers especially. 

Recommendations 

Much more can and needs to be done in developing opportunities within each 
classroom for more writing, more revising, more peer editing, and more sharing 
of writing. Large blocks of time for whole class writing periods need to be 
scheduled on a regular basis — two to three hours a week at the minimum — 
not only as a supplement to some of the woiA in reading but also in place of 
some of the reading skilll work. The teaching of writing by its very nature 
includes the teaching of reading skills, and a greater allocation of time to 
the teaching of writing will improve reading as well as writing skills. Further, 
iiiservice workshops in writing conducted by resources within the school should 
be planned; teachers who have developed effective or interesting ways to stimulate, 
organize, and assess vritin* experiences in their own classrooms should have 
the opportunity to lielp their cclleagixes. Teachers -also need time within the 
school day to share ideas witl each other and to examine, compare, and assess 
student writing regularly with a variety of techniques. As teachers feel en» 
couraged to develop writing-centered classrooms where students write, discuss, 
revise, and share their writing frequently, then listening, speaking, and reading will 
naturally be integrated ~ with a visible product that can be enjoyed by the 
children, their teachers, and their parents. 

Attachment A: Directions for eliciting the writing samples. 

Attachment Bj Criteria for judging the writing samples. 

Attachment Cj Samples of Grade 5 writing from the rating session, with a score 

representing the unanimous judgment of the 3 raters who rated them. 

Attachment Dj Outline of the curriculum guide developed by the Trotter School 
faculty. 

Attachment Ej Correction Symbol System devised by the Trotter School faculty. 
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Attachment A 



To I All Grade 5 teachers 
Fromi Sandra Stotsky 
Date I December 1?, 1979 

About: Samples of writing for the holistic rating session 



Please have all your Grade 5 students write on the following topic as soon 
as possible* 

If Only I Could ... 

As an introduction, please tell the students that Ms. Jackson wants to 
see how well all the children In the school are writing. Assure them 
that their papers will not be graded and that you want them to show 
Ms. Jackson their best writing. 

Have them use white-lined paper. Tell them that they should not place 
their name or any identifying data on these papers . Have them use "If 
Only I Could" as the title and the first k words of the story on their 
paper. Allow about 20 minutes for writing. Please give the papers 
to Michaela Spillane. Thank you for your cooperation. 
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Attachment £ 



Guide Sheet £or Holistic Evaluation of Grade 5 Writing Samples 

William Ma Trotter School 
January 25, 19bO 

A. Paper rated "4" on imaginative writing should: 



1. shov# excellent development of idea, or topic. 

2* show very good sense of closure or climax. 

3- show little redundancy, very good use of details. 

4. show use of metaphor, descriptive, or colorful vocabulary. 

5. show few surface error problems (spelling, usage, 

capitalization, punctuation). 

6. show excellent sentence sense. 



B. Paper rated '•3" on imaginitive writing should show: 



1. good developnent of opening idea or topic. 

2. good sense of closure or climax. 

3. some redundancies, but good use of details. 

4. good use of descriptive or colorful vocabulary. 

5. some surface error problems. 

6. very good sentence sense. 



C. Paper rated "2" on imaginative writing should have: 



1. adequate development o£ opening idea or topic. 

2. some sense of closure or climax. 

3. some details but not have many. 

4. many surface error problems. 

5. some descriptive vocabulary. 

6. problems in sentence sense. 



D. Paper rated "1" on imaginaginative writing should show: 



1. poor development of idea or topic. 

2. poor or no sense of closure or climax. 

3. much redundancy or few, if any details. 

4. many surface error problans, illegibility, and 

5. poor sentence sense. 
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This paper received a rating of 4 from 3 raters. 



\ 1 In^I^^c:^^ ^7.. 77.1.1'' 







I 





A^JL^ jf-*^ 





ERIC 



BBS 



15 



This paper received a rating of 3 
from 3 raters. 



* ..... 



^3 






.1^ 





.jt!^JtJtL—)^2<^. --JiiA^ C^^au^ ^^S ^-rAs^--^s>'?.£>"«^^^ 

-/yUli "fti/WUtM— C-a^ ~-^ /r •;;;i5 — 

: C^r^__ 





ERIC 



BEST 



16 



1> 



This paper received a rating of 3 from 3 raters. 



9> 






4 







! : • This :^per receiv^ta. a ■ wi1t?:rig of • 3 i'rpm. 3 raters . ' " , 




SljOJ^S~~IJ^i--'->^-^''^~^~^-''^ C (] t 



1 



ERIC 



5^ i,<j-rTJi wiior'^ . ^'^ - ■ I 

J?v^dU-.-o<ncl - -ojisi- yluJ:.. ^ 



1 

• 1 
j.. 

I 
I 



BEST ay;.:'-": gc?t 



P-^ This paper received a rating of 2 
from 3 raters-. 



-M.. 





/vu,: cA- u/'^LiC ._ life'- . . . ^<?f;^^-„ . 




yyuL f^^^f^^^.-:^^^'' . _ 

_ _ __. ."^liMi"^^"*'^^;^^ ._ . . ._ 



BEST 



This paper received a rating of 2 from 3 raters. 



./y^l^. ^.^...cc^^.jzj^^ - - 

'.^ ,V.i^^.,<:e3aJU^v^ .CLOXjJlA - <J^-^ c^a^tkuj^ 

^..<prCr- ..^TTiu^ ..^^L^ yJ^j\.JiLydL^-<^-r. 0:fr^A. — 

. ■sP • _Juj-pfl... J ._- ..S^~-- -■ — — — — 



BES'l' rJ:. 



This paper received a rating of 1 from 3 raters. 



"•J -J I ^ r. ^ I 



_ >^t^- . - . . - - - - ■ — : ~~. 1 



This papu.., ^.^^elved a rating of 1 from 3 raters. 



p. 



■3 






Attachment D 



OUTLINE OF CURRICUI.UM GUIDE FOR WRITING 
William M. Trotter School 
1979 



BASIC 


COMPETENCIES 




Grades 










1 


2 


3 








YYVW 


KXXXX 


QCXXX 




iAXXXX 


A 

Am 


«M ^% ^ ^ ^% ap^ « « ^ M« 


X 


X 


X 


Y 
A 


V 

A 


•Q 

O m 


■1 M 4* «^ 4* •! AM 9 


X 


X 


X 


Y 
A 


Y 
A 


n 


^ ^ ^ ^« «ft 




X 


X 


Y 
A 


Y 
A 


"n 

Urn 


W v% ^ A ^ 1* «^ 




t 

i X 


Y 

A 


Y 
A 


r» 
£i* 


C 4* A V* AO 


X 


Y 
A 


1 

1 . 

1 Y 


Y 


Y 

A 


1? 


^1 A n v« ■4 «^ 4* •t AM ^ 




X 


i 

X 


X 


X 


r» 


^1 A ^ ■€ ^% 






X 


V 
A 


V 
A 










X 


Y 
A 


A 


I » 












X 


J. 












X 


K. 




X 


X ■ 


V 

A 


Y 
A 


Y 

A 


L. 






X 


Y 
A 


Y 

A 


Y 

A 


M. 










X 


X 


N. 












X 


0. 
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II. Development of Sentences and Paragraphs 


xxxxx 


ICXXXX 


QCXXX 


xxxxx 


xxxxx 


A. 




xxxxx. 


xxxxx 


QQQK 


xxxxxxxxxx 




1. Use of complete sentences • • . 


X 


X 


X 
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X 










X 


x 
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3. Elimination of fragments. . . . 
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B. 
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2, Using supporting sentences. . . 








X 


x 




3. Using a concluding sentence • • 








X 


x 



BASIC COMPETENCIES 

III. Mechanics 

A. Capitalization 

1. Capital letter at beginning 

of sentence 

2. Capital letter at beginning of 
proper name (including days of 
week, months of year, place names) 

3. Capital letter for I^ , 

A. Capital letter for important 

words in titles 

5. Capital letters for parts of a 

letter 

B. Punctuation 

1. End punctuation . 

2. Internal punctuation 

a. Comma . 

(1) Dates ~ Addresses • . . 

(2) Elements in a series. • . 

(3) Direct address. . 

(A) Parts of a letter . ., . . 

b. Quotation marks. . . 

c. Apostrophe . ^ 

(1) In contractions 

(2) To show possession. . . . 

d. Period for abbreviated titles 

C. Indentation for paragraphing 

D. Spelling appropriate to ability 

and topic 

IV. Revision 

O Proofreading 
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BASIC COMPETENCIES 

VI. Sentence Analysis 

A. Sentence parts 

1. Subject 

2. Predicate 

*B. Typefe of sentence 

1. Declarative. • . 

2. Interrogative. . • . * 

3. Imperative 

4. Exclamatory 

*C. Parts of speech. . . * 

1. Nouns. . « ^ 

2. Pronouns ...... 

3. Verbs 

4. Adjectives 

5 • Adverbs 
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*Use of formal terminology is encouraged at primary grade levels but the 
teaching of formal terms is not recommended until suggested grade levels. 



Attechment E 



William M, Trotter School 
November, 1979 



GORi?ECTION SYMBOLS 



Grades 1-3 

C a capital letter needed 

C « capital letter not needed 

. a period needed 

K - period not needed 

? 3 question mark needed 

X ^ q[uestiQn.%oaik not needed 

/ m words too close together 

/(^^s ntimber of places on line where 
words too close together 

-aB4* 3 and not needed 

P a puncttiation needed 

I a punctuation not needed 
A 3 word missing 
a too many words 

WR = aettejt foriHation (writing) error 



Grades 3-5 

C a capital letter 'needed 

E « capital letter not needed 

p a punctuation needed 

5 3 punctuation not needed 

SP » spelling error 

RO « run-on sentence 

NS 3 not a sentene^e 

^lati^ s and not needed 

--se- s so not needed 

-*he».- =« then not needed 

^ « begin a new paragraph 

U 3 unclear, can*t read, or 
doesn't make sense 

/S 3 word missing 

V ^ too many words 



Teachers in Grades 1-3 may wish to use 
some of their students are able to use 
your students are able to handle. 



symbols from Grades 3-5 if they feel 
them. Use whatever number of symbols 



